Hardware virtualization

ABSTRACT

Embodiments of the present disclosure relate to traffic class management of NVMe (non-volatile memory express) traffic. One or more virtual controllers for at least one host adapter (HA) of a storage device are generated. Each virtual controller is assigned a unique controller identifier (ID) Additionally, one or more input/output (IO) queues for each virtual controller are established. Further, IO workloads are processed via each IO queue.

BACKGROUND

Storage arrays include powerful computing systems that have substantialamounts of storage connected to them. Storage arrays are configured insuch a way that they can present storage to multiple servers, typicallyover a dedicated network. Accordingly, businesses can store and managebusiness data in a central location. For example, business can usestorage arrays to enable employees to access applications, share data,work on files in conjunction with other employees, and store workproduct on the arrays. To enable seamless use of the storage arrays, thestorage arrays can be virtualized for access by each employee'scomputing device. Storage virtualization involves presenting a logicalview of the storage array's physical storage to each employee'scomputing device.

SUMMARY

Embodiments of the present disclosure relate to traffic class managementof NVMe (non-volatile memory express) traffic. One or more virtualcontrollers for at least one host adapter (HA) of a storage device aregenerated. Each virtual controller is assigned a unique controlleridentifier (ID) Additionally, one or more input/output (IO) queues foreach virtual controller are established. Further, IO workloads areprocessed via each IO queue.

In embodiments, each virtual controller can be provisioned with one ormore storage device resources.

In embodiments, each virtual controller can be provided with one or morestorage device resources

In embodiments, a distinct connection ID can be established for each IOqueue

In embodiments, each IO queue can be assigned a service level (SL) thatdefines one or more expected performance metrics for each IO operationqueued in each IO queue.

In embodiments, at least one performance metric can correspond to arange of IO operations per second (IOPS).

In embodiments, each IO queue can be provisioned with the one or morestorage device resources based on each queue's assigned SL.

In embodiments, an IO queue index can be established for each IO queue.The IO queue index can identify one or more of: IO operations queued ineach IO queue, a queue depth, queue total size, and a next availablequeue memory slot, amongst other queue related information.

In embodiments, communication channels can be established with one ormore host devices.

In embodiments, each host device can be provided with informationcorresponding to each virtual controller and the one or more IO queuesestablished for each virtual controller. The information can enable thehost device to direct each IO operation to one of the virtualcontrollers and a particular IO queue based on the IO queue's assignedSL.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, features and advantages will beapparent from the following descriptions of the embodiments, asillustrated in the accompanying drawings in which like referencecharacters refer to the same parts throughout the different views. Thedrawings are not necessarily to scale, emphasis instead being placedupon illustrating the principles of the embodiments.

FIG. 1 is a block diagram of an example a storage system in accordancewith example embodiments disclosed herein.

FIG. 2 includes a block diagram of n host adapter in accordance withexample embodiments disclosed herein.

FIG. 3 is a flow diagram of an example method for managing NVMe trafficin accordance with example embodiments disclosed herein.

FIG. 4 is a flow diagram of an example method for processing NVMetraffic in accordance with example embodiments disclosed herein.

DETAILED DESCRIPTION

Data has become a key input for driving growth, enabling businesses todifferentiate themselves and support a competitive edge. For instance,businesses can harness data to make decisions about finding newcustomers, increasing customer retention, improving customer service,making better marketing decisions, and predicting sales trends, amongstothers. Businesses both generate and collect vast amounts of data andoften store the data in a storage array.

A storage array is a storage system that allows an organization toensure timely delivery of data to end users (e.g., employees),applications, and other information technology (IT) systems over acommunications network (e.g., a storage area network (SAN)). A storagearray can use a disk storage protocol to deliver block access storageservices. For example, block access storage services enableorganizations to manage access to critical information stored by one ormore disk drives of the storage array. The protocols can include storagearray can include multiple disk drives to store, e.g., data andapplications. Example protocols can include Fibre Channel, SmallComputer Systems Interface (iSCSI), Serial Attached SCSI (SAS), andFibre Connection (FICON), amongst others, which were originally designedfor hard disk drive (HHD) systems.

Because HDDs rely on spinning disks, motors, and read/write heads, usingmagnetism to store data on a rotating platter, they are prone tobreaking down. As such, organizations are increasingly requestingvendors to supply storage arrays with flash-based solid-state drives(SSDs), which do not include moving parts. However, current disktransfer protocols cannot maximize the performance capabilities of SSDs.Thus, vendors have started to design storage arrays that include an NVMe(Non-Volatile Memory Express) architecture. NVMe is a high-performancenon uniform memory access (NUMA) interface protocol built on peripheralcomponent interconnect express (PCIe), an interface standard forconnecting high-speed components. Accordingly, NVMe allows storagearrays to use the full capabilities of SSDs to increase performancemetrics (e.g., response times).

Embodiments of the present disclosure extend these connectivityprimitives to map an input/output (IO) operation of a workload to aspecific storage array sub-system. For example, the embodimentsimplement one or more protocols for traffic class management of NVMe(non-volatile memory express) traffic. An embodiment can generate one ormore virtual controllers for at least one host adapter (HA) of a storagedevice. Each virtual controller can be assigned a unique controlleridentifier (ID) The embodiment can also establish one or moreinput/output (IO) queues for each virtual controller are established.Further, the embodiment can process IO workloads via each IO queue.

Regarding FIG. 1 , a system 10 includes a data storage system 12connected to host systems 14 a-n through communication medium 18. Inembodiments, the hosts 14 a-n can access the data storage system 12, forexample, to perform input/output (IO) operations or data requests. Thecommunication medium 18 can be any one or more of a variety of networksor other type of communication connections as known to those skilled inthe art. The communication medium 18 can be a network connection, bus,and/or other type of data link, such as a hardwire or other connectionsknown in the art. For example, the communication medium 18 can be theInternet, an intranet, network (including a Storage Area Network (SAN))or other wireless or other hardwired connection(s) by which the host 14a-n can access and communicate with the data storage system 12. Thehosts 14 a-n can also communicate with other components included in thesystem 10 via the communication medium 18.

Each of the hosts 14 a-n and the data storage system 12 can be connectedto the communication medium 18 by any one of a variety of connections ascan be provided and supported in accordance with the type ofcommunication medium 18. The processors included in the hosts 14 a-n canbe any one of a variety of proprietary or commercially available singleor multi-processor system, such as an Intel-based processor, or othertype of commercially available processor able to support traffic inaccordance with each embodiment and application.

It should be noted that the examples of the hardware and software thatcan be included in the data storage system 12 are described herein inmore detail and can vary with each embodiment. Each of the hosts 14 a-nand data storage system 12 can all be located at the same physical siteor can be in different physical locations. Examples of the communicationmedium 18 that can be used to provide the different types of connectionsbetween the host computer systems and the data storage system of thesystem 10 can use a variety of different communication protocols such asNVMe, and the like. Some or all the connections by which the hosts 14a-n and data storage system 12 can be connected to the communicationmedium can pass through other communication devices, such switchingequipment 105 that can exist such as a phone line, a repeater, amultiplexer or even a satellite.

Each of the hosts 14 a-n can perform distinct types of data operationsbased on task types. In embodiments, any one of the hosts 14 a-n canissue a data request to the data storage system 12 to perform a dataoperation. For example, an application executing on one of the hosts 14a-n can perform a read or write operation resulting in one or more datarequests to the data storage system 12.

Although system 12 is illustrated as a single data storage array, askilled artisan understands that the system 12 can also represent, forexample, multiple data storage arrays alone, or in combination with,other data storage devices, systems, appliances, and/or componentshaving suitable connectivity, such as in a SAN. Further, an embodimentcan include data storage arrays or other components from one or morevendors.

The data storage system 12 can include a plurality of data storagedevices, such as disks 16 a-n. The disks 16 a-n can include one or moreone or more storage media drives and/or one or more solid state drives(SSDs). An SSD is a data storage device that uses solid-state memory tostore persistent data. An SSD using SRAM or DRAM, rather than flashmemory, is an example of a RAM drive. SSD can refer to solid stateelectronics devices as distinguished from electromechanical devices,such as hard drives, having moving parts. Flash devices or flash memorybased SSDs are one type of SSD that do not include moving parts.

The data storage array 12 can include several types of adapters ordirectors, such as an HA 21 (host adapter), RA 40 (remote adapter),and/or device interface 23. Each of the adapters HA 21, RA 40 caninclude hardware having a processor that can execute operations usingcode stored on local memory. The HA 21 can manage communications anddata operations between one or more host systems 14 a-n and the globalmemory (GM) 25 b. In an embodiment, the HA 21 can be a Fibre ChannelAdapter (FA) or another adapter which helps host communications. Thesystem 12 can have a design in which the HA 21 is at the system'sfront-end.

Further, the data storage array 12 can include one or more RAs (e.g., RA40) that can enable communications between data storage arrays. The datastorage array 12 can include one or more device interfaces 23 thatenable data transfers to/from the data storage devices 16 a-n. The datastorage interface 23 can also include device interface modules, such asa disk adapter (DAs 30 (e.g., storage media controller), flash driveinterface 35, and the like. The system 12 can have the interface 23, DA30 and drive interface 35 as part of its back-end architecture, whichinterfaces with the disks 16 a-n via, e.g., a physical link 115.

One or more internal logical communication paths can exist between thedevice interfaces 23, the RAs 40, the HAs 21, and the memory 26. Anembodiment, for example, can use one or more internal busses and/orcommunication modules. For example, the global memory 25 b can be usedto transfer data and other communications between the device interfaces,HAs and/or RAs in a data storage array. In one embodiment, the deviceinterfaces 23 can perform data operations using a cache that can includein the global memory 25 b, for example, when communicating with otherdevice interfaces and other components of the data storage array. Theportion 25 a is a memory that can be used in connection with otherdesignations that can vary in accordance with each embodiment.

Host systems 14 a-n can issue input/output (IO) operations throughmedium 18 that include, e.g., a data read or write requests. The hostsystems 14 a-n can access data stored on the disks 16 a-n via one ormore logical volumes (LVs). In embodiments, the HA 21 can manage thetraffic received from the hosts 14 a-n and the LVs. In embodiments, theHA 21 can include a traffic processor 110 that manages NVMe trafficreceived from the hosts 14 a-n as described in greater detail in thefollowing paragraphs.

Regarding FIG. 2 , a storage area network (SAN) 200 includes a storagearray 12 that is communicatively coupled to hosts 14 a-n. The SAN 200can be adapted to enable a connectivity primitive such as NVMe.Accordingly, the array 12 and hosts 14 a-n can implement an NVMecommunication protocol to send and receive 10 workloads 240. Further,the array 12 can include a HA 21 and devices 230 a-n. Although theembodiment illustrated by FIG. 2 illustrates a single HA, a skilledartisan understands that the array 12 can include a plurality of HAs.The devices 230 a-n can be any hardware and/or software resource of thearray 12. For example, the devices 230 a-n can be substantially likecomponents 23, 25 a-b, 26, 30, 35, and 16 a-n, amongst other knowncomponents of storage arrays as illustrated in FIG. 1 .

In embodiments, the HA 21 can include a traffic processor 110 thatmanages the IO workloads 240 received from the hosts 14 a-n. The IOworkloads 240 can include IO operations 215 a-n such as read and writerequests, amongst other known IO operations and SAN communications.

The processor 110 can establish a controller 205 that directs the IOoperations 215 a-n to certain storage array resources (e.g., devices 230a-n). In embodiments, the processor 110 can establish one or morecontrollers 205. For example, the processor 110 can establish a singlecontroller 205 that controls workloads 240 received by the HA 21 and anyother HA of the array 12. In other examples, the processor 110 canestablish a set of controllers 205 for each HA 21. Further, theprocessor 110 can establish each controller as a virtual device that isallocated one or more hardware and/or software resources of the array12.

In embodiments, the processor 110 can assign each established controller205 a distinct controller identifier (ID). In an example, the processor110 can enable each controller to manage access to one or more of thedevices 230 a-n by the IO operations 215 a-n. In further examples, theprocessor 110 can establish each controller with a distinct controllerID. Using any ID generation technique, the processor 110 can establisheach controller ID based on the devices 230 a-n each controller 205 isenabled to manage. The processor 110 can generate a searchablecontroller data structure that maps each controller to correspondingconnection IDs and assigned devices 230 a-n

In further embodiments, the controller 205 can establish one or more IOqueues 210 a-n. Each IO queue 210 a-n can receive, and buffer IOoperations 215 a-n included in the workload 240. In examples, thecontroller 205 can designate each queue to buffer one or more of thecontroller managed devices 230 a-n. In examples, the controller 205 canassign each queue 210 a-n a distinct connection ID. Each queue'sdistinct connection ID can be based on each queue's assigned queuemanaged controller devices 230 a-n. Further, each queue 210 a-n cangenerate an IO queue index that identifies, e.g., a capacity of eachqueue 210 a-n, address (e.g., location) of a next available memory slotthat can buffer an incoming IO operation. Additionally, the controllercan generate a searchable queue data structure that maps each queue to acorresponding connection ID and assigned devices 230 a-n.

In embodiments, the hosts 14 a-n and array 12 can communicatively coupleusing one or more initialization techniques (e.g., login techniques).During initialization, the hosts 14 a-n can issue commands (e.g.,identify commands) and the controller 205 and/or queues 210 a-n canbroadcast their respective controller and connection IDs. Accordingly,each host 14 a-n can determine the array 12 to which the controller 205belongs and the devices 210 a-b each controller 205 manages using one orboth controller and connection IDs. As such, the controller ID andconnection ID enable quick and efficient communication that can increasestorage array performance (e.g., response times).

For example, each controller ID can include information related to thearray (e.g., board and port numbers, array serial number, and system ID,amongst other related array information). Additionally, each connectionID can include information such as a director data, port data, subsystemID (e.g., an ID corresponding to the queue managed controller devices230 a-n) and the IO queue index, amongst other relevant information.Thus, the controller ID and connection ID allow each host 14 a-n todirect IO operations to a specific queue based on a service levelagreement (SLA). The SLA can define performance requirements (e.g.,service levels) for each type of IO operation, application source of anIO operation, and user of each host 14 a-n, amongst other SLA relatedfactors.

For example, each application used by an organization can have differentbusiness values. As such, data requests (e.g., IO operations)corresponding to each application can have a traffic prioritycorresponding to each application's business value. Duringinitialization of a storage array (e.g., array 12), the organization candefine quality of service (QoS) (e.g., expected performance envelope)levels for each application. The QoS levels can be grouped based ontheir respective performance envelopes (e.g., range of input/outputoperations per second (IOPS)). Each group can be assigned a servicelevel (SL) having designations such as DIAMOND, GOLD, SILVER, AND BRONZEthat each define the expected array performance for an IO operationincluding the designation.

In embodiments, the processor 110 can use one or more machine learning(ML) techniques to anticipate IO workloads and corresponding performancerequirements (e.g., service levels (SL)) of each IO operation includedin the anticipated workloads. Based on the anticipated IO workloads, theprocessor 110 can configure each controller 205 to manage certain IOSLs. For example, the processor 110 can provision the controller 205with resources required to meet specific IO SLs. In embodiment, theresources can correspond to a number and/or type of device of thedevices 230 a-n. Further, the controller 205 can configure each of thequeues 210 a-n to buffer certain IO SLs. As such, the controller 205 canallocate portions of its allocated resources to each queue 210 a-n tomeet specific IO SL requirements.

Further, the controller 205 can be configured to determine a status ofeach queue 210 a-n. For example, the controller 205 can determineavailable index elements (e.g., elements I/O-0-N-n) of each queue 210a-n. Using the determined status, the controller 205 can issue an updatesignal each host 14 a-n. The update signal can include informationrelated to available queue elements of the queues 210 a-n and schedulinginstructions for issuing 10 operations to one or more of the queues 210a-n.

In further embodiments, each host 14 a-n can include one or more of theconnection ID and controller ID to each IO operation to direct each IOoperation in the workload 240 to a specific queue and controller basedon IO SL requirements. Additionally, each host 14 a-n can select aconnection ID and controller ID to include with an IO operation based onthe information contained in the update signal.

The following text includes details of one or more methods and/or flowdiagrams in accordance with this disclosure. For simplicity ofexplanation, the methods are depicted and described as a series of acts.However, acts in accordance with this disclosure can occur in variousorders and/or concurrently, and with other acts not presented anddescribed herein. Furthermore, not all illustrated acts can be requiredto implement the methods in accordance with the disclosed subjectmatter.

Regarding FIG. 3 , a method 300 can be executed by a device, e.g., thetraffic class processor 110 of FIG. 1 . The method 300, at 310, caninclude generating one or more virtual controllers for at least one hostadapter (HA) of a storage device. Each virtual controller can beassociated with a unique controller identifier (ID). In embodiments, themethod 300, at 310, can also include establishing one or moreinput/output (IO) queues for each virtual controller. The method 300, at315, can include determining a service level (SL) corresponding to eachof the one or more IO operations. In embodiments, the method 300, at315, can further include processing IO workloads via each IO queue.

It should be noted that the method 300 can be performed according to anyof the embodiments described herein, known to those skilled in the art,and/or yet to be known to those skilled in the art.

Regarding FIG. 4 , a method 400 can be executed by a device, e.g., thetraffic processor 110 of FIG. 1 . The method 400, at 410, can includeprovisioning each virtual controller with one or more storage deviceresources. The device interface can include, e.g., the DA and trafficclass processor. In embodiments, the method 400, at 410, can alsoproviding each IO queue with access to the one or more storage deviceresources provisioned to the IO queue's respective virtual controller.Further, the method 400, at 415, can include establishing a distinctconnection ID for each IO queue.

It should be noted that the method 400 can be performed according to anyof the embodiments described herein, known to those skilled in the art,and/or yet to be known to those skilled in the art.

Methods and/or flow diagrams are illustrated with this disclosure forsimplicity of explanation. The methods are depicted and described as aseries of acts. However, acts in accordance with this disclosure canoccur in various orders and/or concurrently, and with other acts notpresented and described herein. Furthermore, not all illustrated actscan be required to implement the methods in accordance with thedisclosed subject matter.

The above-described systems and methods can be implemented in digitalelectronic circuitry, in computer hardware, firmware, and/or software.The implementation can be as a computer program product. Theimplementation can, for example, be in a machine-readable storagedevice, for execution by, or to control the operation of, dataprocessing apparatus. The implementation can, for example, be aprogrammable processor, a computer, and/or multiple computers.

A computer program can be written in any form of programming language,including compiled and/or interpreted languages, and the computerprogram can be deployed in any form, including as a stand-alone programor as a subroutine, element, and/or other unit suitable for use in acomputing environment. A computer program can be deployed to be executedon one computer or on multiple computers at one site.

Method steps can be performed by one or more programmable processorsexecuting a computer program to perform functions of the conceptsdescribed herein by operating on input data and generating output.Method steps can also be performed by and an apparatus can beimplemented as special purpose logic circuitry. The circuitry can, forexample, be a FPGA (field programmable gate array) and/or an ASIC(application-specific integrated circuit). Subroutines and softwareagents can refer to portions of the computer program, the processor, thespecial circuitry, software, and/or hardware that implement thatfunctionality.

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any digital computer. A processor receivesinstructions and data from a read-only memory or a random-access memoryor both. The essential elements of a computer are a processor forexecuting instructions and one or more memory devices for storinginstructions and data. A computer can include, can be operativelycoupled to receive data from and/or transfer data to one or more massstorage devices for storing data (e.g., magnetic, magneto-opticalstorage media, or optical storage media).

Data transmission and instructions can also occur over a communicationsnetwork. Information carriers suitable for embodying computer programinstructions and data include all forms of non-volatile memory,including by way of example semiconductor memory devices. Theinformation carriers can, for example, be EPROM, EEPROM, flash memorydevices, magnetic storage media, internal hard storage media, removablestorage media, magneto-optical storage media, CD-ROM, and/or DVD-ROMstorage media. The processor and the memory can be supplemented by,and/or incorporated in special purpose logic circuitry.

To provide for interaction with a user, the above described techniquescan be implemented on a computer having a display device. The displaydevice can, for example, be a cathode ray tube (CRT) and/or a liquidcrystal display (LCD) monitor. The interaction with a user can, forexample, be a display of information to the user and a keyboard and apointing device (e.g., a mouse or a trackball) by which the user canprovide input to the computer (e.g., interact with a user interfaceelement). Other kinds of devices can be used to provide for interactionwith a user. Other devices can, for example, be feedback provided to theuser in any form of sensory feedback (e.g., visual feedback, auditoryfeedback, or tactile feedback). Input from the user can, for example, bereceived in any form, including acoustic, speech, and/or tactile input.

The above described techniques can be implemented in a distributedcomputing system that includes a back-end component. The back-endcomponent can, for example, be a data server, a middleware component,and/or an application server. The above described techniques can beimplemented in a distributing computing system that includes a front-endcomponent. The front-end component can, for example, be a clientcomputer having a graphical user interface, a Web browser through whicha user can interact with an example implementation, and/or othergraphical user interfaces for a transmitting device. The components ofthe system can be interconnected by any form or medium of digital datacommunication (e.g., a communication network). Examples of communicationnetworks include a local area network (LAN), a wide area network (WAN),the Internet, wired networks, and/or wireless networks.

The system can include clients and servers. A client and a server aregenerally remote from each other and typically interact through acommunication network. The relationship of client and server arises bycomputer programs running on the respective computers and having aclient-server relationship to each other.

Packet-based networks can include, for example, the Internet, a carrierinternet protocol (IP) network (e.g., local area network (LAN), widearea network (WAN), campus area network (CAN), metropolitan area network(MAN), home area network (HAN)), a private IP network, an IP privatebranch exchange (IPBX), a wireless network (e.g., radio access network(RAN), 802.11 network, 802.16 network, general packet radio service(GPRS) network, HiperLAN), and/or other packet-based networks.Circuit-based networks can include, for example, the public switchedtelephone network (PSTN), a private branch exchange (PBX), a wirelessnetwork (e.g., RAN, Bluetooth, code-division multiple access (CDMA)network, time division multiple access (TDMA) network, global system formobile communications (GSM) network), and/or other circuit-basednetworks.

The transmitting device can include, for example, a computer, a computerwith a browser device, a telephone, an IP phone, a mobile device (e.g.,cellular phone, personal digital assistant (PDA) device, laptopcomputer, electronic mail device), and/or other communication devices.The browser device includes, for example, a computer (e.g., desktopcomputer, laptop computer) with a world wide web browser (e.g.,Microsoft® Internet Explorer® available from Microsoft Corporation,Mozilla® Firefox available from Mozilla Corporation). The mobilecomputing device includes, for example, a Blackberry®.

Comprise, include, and/or plural forms of each are open ended andinclude the listed parts and can include additional parts that are notlisted. And/or is open ended and includes one or more of the listedparts and combinations of the listed parts.

One skilled in the art will realize the concepts described herein can beembodied in other specific forms without departing from the spirit oressential characteristics thereof. The foregoing embodiments aretherefore to be considered in all respects illustrative rather thanlimiting of the concepts described herein. Scope of the concepts is thusindicated by the appended claims, rather than by the foregoingdescription, and all changes that come within the meaning and range ofequivalency of the claims are therefore intended to be embraced therein.

What is claimed is:
 1. A method comprising: generating one or morevirtual controllers for at least one host adapter (HA) of a storagearray, wherein each virtual controller is assigned a unique controlleridentifier (ID); establishing one or more input/output (TO) queues foreach virtual controller; allocating a portion of the one or more virtualcontroller's resources to the one or more controller's IO queues basedon IO service levels (SLs) related to the one or more IO queues, whereinthe resources include memory and storage devices; providing each IOqueue with metadata that identifies the IO queue's queue depth, totalsize, and available memory slots; based on each IO queue' status,issuing an update IO queue status signal to each remote hostcorresponding to each IO queue; and processing IO workloads via the oneor more IO queues.
 2. The method of claim 1 further comprising:provisioning each virtual controller with one or more storage arrayresources.
 3. The method of claim 2 further comprising: providing eachIO queue with access to the one or more storage array resourcesprovisioned to the IO queue's respective virtual controller.
 4. Themethod of claim 2 further comprising: establishing a distinct connectionID for each IO queue.
 5. The method of claim 2 further comprising:associating each IO queue with a service level (SL) that defines one ormore expected performance metrics for each IO operation queued in eachIO queue.
 6. The method of claim 5, wherein at least one performancemetric corresponds to a range of IO operations per second (IOPS).
 7. Themethod of claim 6 further comprising: provisioning each IO queue withthe one or more storage array resources based on each queue's assignedSL.
 8. The method of claim 5 further comprising: establishingcommunication channels with one or more host devices.
 9. The method ofclaim 5 further comprising: providing each host device with informationcorresponding to each virtual controller and the one or more IO queuesestablished for each virtual controller, wherein the information enablesthe host device to direct each IO operation to one of the virtualcontrollers and a particular IO queue based on the IO queue's assignedSL.
 10. An apparatus configured to at least one processor configured to:generate one or more virtual controllers for at least one host adapter(HA) of a storage array, wherein each virtual controller is assigned aunique controller identifier (ID); establish one or more input/output(TO) queues for each virtual controller; allocate a portion of the oneor more virtual controller's resources to the one or more controller'sIO queues based on TO service levels (SLs) related to the one or more IOqueues, wherein the resources include memory and storage devices;provide each IO queue with metadata that identifies the IO queue's queuedepth, total size, and available memory slots; based on each IO queue'status, issue an update IO queue status signal to each remote hostcorresponding to each IO queue; and process TO workloads via the one ormore IO queues.
 11. The apparatus of claim 10 further configured to:provision each virtual controller with one or more storage arrayresources.
 12. The apparatus of claim 11 further configured to: provideeach IO queue with access to the one or more storage array resourcesprovisioned to the IO queue's respective virtual controller.
 13. Theapparatus of claim 11 further configured to: establish a distinctconnection ID for each IO queue.
 14. The apparatus of claim 11 furtherconfigured to: associate each IO queue with a service level (SL) thatdefines one or more expected performance metrics for each TO operationqueued in each IO queue.
 15. The apparatus of claim 14, wherein at leastone performance metric corresponds to a range of IO operations persecond (IOPS).
 16. The apparatus of claim 15 further configured to:provision each IO queue with the one or more storage array resourcesbased on each queue's assigned SL.
 17. The apparatus of claim 14 furtherconfigured to: establish communication channels with one or more hostdevices.
 18. The apparatus of claim 14 further configured to: provideeach host device with information corresponding to each virtualcontroller and the one or more IO queues established for each virtualcontroller, wherein the information enables the host device to directeach IO operation to one of the virtual controllers and a particular IOqueue based on the IO queue's assigned SL.